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DIGITAL  DATA  LOGGING  AND 
PROCESSING 

Derbyshire  survey,  1997 


1.  INTRODUCTION 


In  1997,  the  Deep  Submergence  Group  (DSG)  of  the  Woods  Hole  Oceanographic 
Institution  (WHOI)  surveyed  the  wreckage  field  of  the  M.V.  Derbyshire.  The  motivation 
for  the  survey  and  its  results  are  described  elsewhere  (Williams  et  al,  1998).  The  purpose 
of  this  report  is  to  describe  the  digital  data  logging  and  processing  systems  that  were  used 
by  the  Deep  Submergence  Group  during  the  survey.  The  report  is  divided  into  four 
sections:  this  Introduction,  a  description  of  the  collection  mechanisms,  a  description  of  the 
processing  schemes  and  series  of  appendices.  The  appendices  include  a  glossary  of 
terms,  a  description  of  data  formats,  and  a  comparison  of  electronic  still  camera 
processing  choices.  Readers  desiring  information  on  the  equipment  used,  on  the 
operations,  or  on  the  analysis  effort  performed  by  the  on-board  Inspection  and 
Verification  (I  &  V)  Team  or  by  the  Assessors  ashore  are  directed  to  (Williams  et  al,  1998), 
(Ballard,  1993)  and  (Bowen,  et  al,  1993). 
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2.  DATA  COLLECTION 


Three  different  underwater  vehicles  were  used  during  the  survey,  the  DSL-120  towed 
sonar,  the  Argo  II  towed  imaging  sled,  and  the  Jason  ROV.  Due  to  the  differing  natures  of 
the  vehicle’s  telemetry  systems  and  sensors,  there  were  differences  in  logging  and 
processing  methodologies. 


2.1.  SERIAL  DATA  LOGGING 

For  the  purposes  of  this  report,  serial  data  is  defined  as  that  data  which  is  usually 
represented  by  a  time  series  and  is  sent  to  the  data  logging  systems  as  a  single  stream  of 
digital  bits.  It  is  usually  sent  via  RS-232,  a  transmission  standard  that  is  defined  for  inter¬ 
device  and  inter  computer  communications.  Serial  data  logging  can  be  contrasted  to 
electronic  still  camera  data  logging  or  sonar  data  logging  primarily  by  the  nature  of  the 
data  being  transmitted. 

Figure  1  shows  a  block  diagram  of  the  serial  data  logging  system  used  for  the 
Derbyshire  survey.  (Note  that  although  some  of  the  particular  details  of  hardware  types, 
etc,  have  changed,  the  current  DSG  unmanned  vehicles  serial  data  logging  scheme 
remains  virtually  identical  to  that  shown.  The  figure  describes  the  flow  of  discrete  pieces 
of  information,  encoded  as  ASCII  text,  between  a  number  of  computer  systems.  Other 
types  of  data,  such  as  electronic  still  camera  (ESC)  images  and  sonar  records  are  handled 
differently  and  will  be  described  in  later  sections  of  this  report.  Figure  1  applies  to  all  of 
the  vehicle  systems  used  during  the  survey. 

The  thin  black  lines  in  Figure  1  represent  RS-232  communications,  and  are  symbolic  in 
nature.  The  actual  wiring  paths  run  through  a  patch  bay  in  the  control  van  to  simplify 
routing  and  connections.  However,  the  basic  signal  flow  for  most  data  logging  can  be 
represented  by  a  line  from  the  sending  computer  to  a  device  called  a  Portmaster.  The 
Portmaster  multiplexed  the  RS-232  data  into  packets,  which  are  sent  out  on  an  Ethernet, 
represented  by  the  thick  black  line.  (This  thick  line  is  also  symbolic,  as  the  actual 
network  was  divided  into  segments  with  switches  and  hubs,  to  isolate  the  data  network 
from  the  rest  of  the  shipboard  traffic. 
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Figure  1:  Serial  Data  Logging  System 

The  data  logging  computers  registered  with  the  Portmaster  to  receive  all  data  coming 
in  on  a  particular  serial  port.  They  time  stamped  the  data  if  necessary,  logged  it  to  hard 
disk,  and  distributed  it  (via  network  broadcast)  to  other  shipboard  computers  (such  as  the 
sonar  system). 

A  Chronolog  time  standard  was  used  to  insure  synchronization  of  all  the  computers  in 
the  data  system.  The  Chronolog  was  set  to  Global  Positioning  System  (GPS)  time  as 
collected  at  the  Navigation/Control  Computer.  All  other  systems,  including  the  Jason/Argo 
topside  computer  and  the  data  loggers  were  automatically  set  to  the  Chronolog  on  startup, 
and  periodically  throughout  the  survey.  The  reset  interval  varied  between  systems,  but 
was  almost  always  less  than  1  hour,  making  any  drift  in  the  time  reference  negligible. 

The  data  logging  computers  (Sun  Sparc  LX  workstations)  were  used  for  routine 
monitoring  of  the  serial  data  streams  during  the  Argo  II  and  Jason  portions  of  the  survey. 
Data  logging  watchstanders  were  trained  to  periodically  check  the  status  and  size  of  the 
active  files,  and  to  contact  DSG  personnel  in  the  event  of  any  anomalies.  (Note:  the 
watchstanders  were  also  responsible  for  monitoring  and  changing  video  and  ESC  tapes. 
Due  to  the  exceptionally  high  workload  from  video  systems,  two  watchstanders  were 
employed  for  much  of  the  Derbyshire  survey.) 

The  DSL-120  topside  computer,  also  a  Sparc  workstation,  was  responsible  for  logging 
DSL-120  attitude  during  that  portion  of  the  survey.  The  two  LX  workstations  were  manned 
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during  this  survey,  since  they  were  responsible  for  collecting  navigation  data  as  a  backup 
to  the  sonar  workstation  recording. 

The  largest  source  of  serial  data  during  the  Argo  II  and  Jason  portions  of  the 
Derbyshire  survey  was  the  vehicle  topside  computer.  Frequent  status  messages,  thruster 
command  values,  and  vehicle  attitude  (pitch,  roll,  and  heading)  data  flowed  continually 
from  topside  to  the  data  logger,  at  a  rate  of  approximately  70  megabytes  per  day.  The 
navigation  system  was  also  a  continuous  source  of  data,  producing  approximately  35 
megabytes  per  day.  All  of  the  serial  digital  data  was  archived  and  processed  daily;  a  later 
section  of  this  report  will  describe  those  steps  in  some  detail. 

2.2.  ELECTRONIC  STILL  CAMERA  LOGGING 

Electronic  Still  Camera  data  was  collected  on  a  Marine  Imaging  Systems  Inc.  (MIS) 
(now  Imetrix  Inc.)  deck  box  every  13  seconds  during  Argo  II  operation,  and  recorded 
onto  exabyte  8mm  tape.  Each  image  was  stored  as  a  576  x  384  pixel  MIS  format  file,  in 
which  each  pixel  is  represented  by  two  bytes.  Part  of  the  MIS  format  is  a  time  stamp, 
which  indicates  the  moment  when  the  shutter  opened  and  the  strobe  fired  exposing  the 
Charge  Coupled  Device  (CCD)  imaging  chip.  Upon  acquisition,  the  raw  image  data  was 
globally  histogram  equalized  to  enhance  viewing,  converted  to  eight-bit  format,  and 
displayed  using  the  graphics  output  board  on  the  topside  deck  box.  The  output  of  the 
video  board  was  NTSC  video,  suitable  for  display  on  any  of  the  monitors  in  the  control 
van  or  throughout  the  ship.  The  raw  MIS  file  was  written  by  the  deck  box  onto  tape  in  a 
Unix  “tar”  format  in  separate  files  of  fifty. 

The  deck  box  was  modified  by  WHOI  personnel  to  send  a  pulse  to  a  Sony  CRV  laser 
disk  recorder,  which  saved  a  frame  of  the  ESC  video  output  for  later  review.  This  laser 
frame  was  saved  with  a  SMPTE  time  stamp  inserted  into  an  audio  track,  allowing 
reconstruction  of  the  time  of  collection. 

The  objective  presented  to  the  watchstanders  responsible  for  changing  the  ESC  tapes 
was  to  put  approximately  1000  images  on  each  tape.  In  theory,  each  tape  could  hold 
over  4800  images,  but  the  full  capacity  was  never  used  so  as  to  minimize  the  loss  of  data 
in  the  event  of  a  tape  failure.  One  thousand  images  also  fit  in  fairly  well  with  the  four 
hour  watch  schedule  followed  during  the  survey,  given  the  approximately  13  second 
image  cycle  time.  Watch  standers  were  also  instructed  to  check  with  the  watch  leader 
before  changing  a  tape,  as  tape  changes  (which  took  several  minutes)  necessitated  a  gap 
in  image  coverage.  As  a  consequence,  tape  changes  usually  occurred  at  the  end  of  lines, 
after  the  vehicle  had  been  towed  out  of  the  wreckage  field  and  was  beginning  a  turn.  In 
consequence,  the  average  number  of  images  on  a  tape  was  somewhat  greater  than  1300. 
Complete  imagery  coverage  of  the  terrain  below  the  vehicle  was  maintained  throughout 
the  ESC  tape  changes  by  maintaining  continuous  recording  of  video  camera  data  during 
the  brief  periods  when  the  ESC  was  off  line  for  a  tape  change. 

After  being  removed  from  the  deck  box,  the  ESC  tape  was  carried  to  the  Assessor’s 
Lab,  where  it  was  checked  into  the  data  archive.  By  the  end  of  the  survey,  exactly  100 
raw  tapes  were  collected.  Of  these  100,  one  tape  was  completely  blank,  due  to  a  tape 
error  on  loading.  (The  error  was  noticed  immediately  and  the  tape  changed.  The  blank 
tape  was  maintained  as  part  of  the  data  stream;  since  it  had  been  pre-labeled  and 
numbered,  discarding  it  would  cause  a  discrepancy  between  the  data  archive  and  the  tape 
log  which  was  kept  in  the  control  van.)  A  later  section  of  this  report  will  describe  the 
processing  and  distribution  of  the  ESC  imagery. 
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Collection  of  the  ESC  imagery  was  uneventful,  with  one  exception.  At  midnight  on 
March  31,  when  the  deck  box  clock  should  have  rolled  over  to  April  1st,  it  changed 
instead  to  March  32d.  When  the  system  software  which  reads  the  clock  data  and  time 
stamps  the  imagery  encountered  that  date,  it  considered  it  invalid,  and  instead  of  inserting 
an  erroneous  time  into  the  ESC  record,  it  inserted  a  blank  string.  Fortunately,  the  real¬ 
time  display  also  includes  date  and  time,  and  the  watchstanders  quickly  noticed  the  its 
absence.  They  summoned  DSG  personnel.  Upon  investigation  of  the  problem,  it  was 
discovered  that  the  deck  box  rolls  over  incorrectly  on  all  month  changes.  The  deck  box 
time  was  set  to  June  10th,  a  date  well  beyond  the  planned  end  of  the  survey.  In  later 
processing,  any  image  with  a.  time  stamp  of  June  10th  was  known  to  come  from  this 
erroneous  period,  and  its  date  was  reset  to  April  1st.  The  date  in  the  deck  box  was  reset 
to  the  correct  date  after  the  change  of  day  to  April  2d. 

As  described,  a  laser  disk  was  used  to  record  real-time  ESC  output.  The  time  stamps 
on  these  recordings  were  used  to  reconstruct  the  time  stamps  on  the  small  number  of 
images  that  were  collected  with  a  blank  string  in  place  of  a  date  and  time.  Although  this 
date  change  difficulty  caused  a  flurry  of  activity  aboard  ship,  it  did  not  result  in  the  loss  or 
improper  time-stamping  of  any  data. 

2.3.  SIDE  SCAN  SONAR  DATA  LOGGING 

Side  Scan  sonar  data  from  the  DSL-120  vehicle  system  was  received  via  a  high-speed 
interface  on  a  Sun  workstation.  There  it  was  time  stamped  (The  workstation  was 
synchronized  to  the  Chronolog  time  standard)  and  written  onto  exabyte  tape  in  its  most 
raw  form.  A  first  stage  of  processing,  converting  raw  data  to  amplitudes  and  angles  was 
performed.  A  copy  of  this  once  processed  data  was  logged  across  the  shipboard  Ethernet 
to  hard  disk,  where  it  was  archived  and  made  available  for  later  processing.  Speed  and 
slant-range  corrected  data  were  displayed  in  the  control  van  and  printed  onto  a  hard-copy 
unit. 


3.  DATA  ARCHIVING  AND  PROCESSING 


In  a  manner  similar  to  the  previous  section  on  Data  Collection,  this  section  will  be 
broken  into  several  parts,  containing  information  describing  the  archiving  and  processing 
of  Serial,  or  time  series  data,  ESC  data,  and  sonar  data.  The  processing  was  performed  on 
an  integrated  network  of  Sun  (Unix)  and  Intel  (Windows  NT)  workstations,  connected  to 
each  other  and  to  the  real-time  logging  systems  by  a  high  speed  Ethernet.  Several  Silicon 
Graphics  workstations  were  also  available  for  data  review  and  visualization,  and  several 

Figure  2  shows  the  network  layout  on  the  R.  V  Thompson. 


Figure  2:  R.V.  Thompson  Network  Layout 


Other  than  those  brought  by  DSG,  significant  processing  resources  were  available  on 
board  the  Thompson,  both  those  belonging  to  the  Thompson  and  those  brought  by 
members  of  the  UK/EU  team.  This  report  will  not  describe  those  resources  or  any  efforts 
undertaken  using  them. 

3.1.  SERIAL  DATA  ARCHIVING  AND  PROCESSING 

Serial  data  archiving  happened  on  a  daily  basis,  using  Greenwich  Mean  Time  (GMT) 
midnight  as  a  break  point.  Each  day,  shortly  after  Greenwich  midnight,  the  raw  files 
collected  during  the  previous  day  were  copied  across  the  network  into  a  raw  file  directory 
on  an  off-line  processing  Sun  workstation.  There  they  were  maintained  for  ten  days  or 
more,  with  older  data  being  periodically  removed  to  make  space.  Regular  exabyte  tape 
backups  of  this  raw  data  were  made,  ensuring  that  each  day’s  raw  files  were  written  to 
several  different  exabyte  tapes  as  redundant  backups.  In  addition,  original  raw  data  was 
left  on  the  logging  computer  for  some  time  after  being  copied  over  to  the  raw  off-line 
directory.  This  process  ensured  that  the  raw  data  was  always  available  on-board  ship,  and 
multiple  copies  were  available  post-cruise. 

The  following  file  name  format  was  used  throughout  the  Derbyshire  survey  for  all  raw 
serial  data  from  the  Argo  II  and  Jason  vehicles,  and  for  all  raw  navigation  data: 

•  YYMMDD.drXXX.NNN 

Where: 

•  YYMMDD  indicates  the  date,  in  year,  month,  day  symbology. 

•  “dr”  is  inserted  by  the  real  time  data  logging  computer,  indicating  the  particular 
serial  port  on  the  portmaster  which  provided  the  data.  Due  to  an  inconsistency 
between  normal  unix  serial  port  naming  and  the  “pseudo”  serial  ports  which  the 
Portmaster  uses,  the  particular  port  number  is  missing  from  the  field.  This  has  been 
true  for  Portmaster-based  logging  for  several  years,  and  was  determined  to  not  be  a 
problem.  In  the  event  that  a  backup  data  logging  computer  is  used,  “ds”  is  used 
instead,  since  on  the  backup  logger,  a  different  set  of  pseudo  serial  ports  is  used. 

•  XXX  indicates  the  type  of  data,  and  is  a  string  chosen  by  the  data  logging 
personnel  at  the  beginning  of  operations.  During  the  Derbyshire  survey,  the 
following  strings  were  used: 

♦  NAV:  navigation 

♦  JAS:  Jason  data 

♦  ARG:  Argo  data 

♦  VID:  video  tape  recorder  data 

♦  CRV:  laser  disk  data 

♦  EVT:  event  data 

•  NNN  indicates  the  file  sequence.  Usually,  this  number  is  000,  but  if  daily  files 
were  closed  and  reopened,  files  subsequent  to  the  first  would  have  ascending 
numbers  in  this  position. 
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The  raw  data  form  the  DSL-120  is  contained  within  the  sonar  record  format;  there  are 
no  separate  raw  data  files  from  this  vehicle.  Unless  otherwise  specified,  all  subsequent 
descriptions  of  data  archiving  and  processing  apply  to  data  from  Argo  II  and  Jason  only. 

Raw  data  files  were  compressed  using  a  standard  Unix  utility  before  archiving. 
Compressed  files  have  a  further  “.Z”  suffix  appended  by  the  utility.  All  Unix  operating 
systems  can  use  the  “uncompress”  command  to  access  these  files;  the  standard  Windows 
utility  “Winzip”  can  open  them  as  well. 

Data  processing  also  happened  on  a  daily  basis.  Two  main  types  of  vehicle  data  were 
processed:  Navigation  data,  and  Attitude  data.  Figure  3  is  a  block  diagram  of  daily 
processing,  which  was  a  several  step  evolution: 


Raw  DataArchiving 


;abyte  Tape 


Step  2 


Figure  3-  Daily  Data  Processing  (serial) 
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3.1.1.  NAVIGATION  PROCESSING 


1.  Data  Abstraction.  The  raw  navigation  data  file  contained  many  types  of  navigation 
data,  from  whatever  vehicle  was  being  used  and  from  the  ship.  Appendix  A 
describes  the  types  and  formats  in  detail.  Ship  data  was  not  routinely  processed. 
Vehicle  data  was  abstracted  from  the  raw  file  using  automated  scripts. 

2.  Data  Cleaning.  The  vehicle  data  file  was  checked  (using  automated  scripts)  for 
records  out  of  order,  incomplete  records,  and  extraneous  characters.  Given  the 
simplicity  of  the  path  between  the  navigation  system  and  the  data  logger,  such 
problems  rarely  if  ever  occurred,  but  the  processing  step  remains  as  a  holdover 
from  the  day  when  inter-computer  communication  was  not  as  reliable  as  it  has 
become. 

3.  Wild-Point  Editing:  A  simple  filter  examined  each  navigation  record  to  ensure  that 
the  coordinates  were  within  the  survey  area.  A  median  rejection  filter  was  used 
to  examine  the  data,  checking  each  point  to  be  sure  that  it  didn’t  differ  from  the 
median  of  its  neighbors  by  more  than  a  specified  amount.  A  velocity  filter  was 
used  to  make  sure  that  the  navigation  records  don’t  indicate  the  vehicle  travelling 
at  an  unreasonable  speed. 

4.  Graphical  Editing.  The  navigation  data  set  was  plotted  in  a  custom  graphical 
editor,  both  in  an  X/Y  geographic  plot,  and  in  various  time  series  plots  (X,  Y,  and 
Z  versus  time).  Navigation  points  thought  to  be  spurious  were  removed  during 
this  process. 

5.  Data  Reorganization.  The  fully  processed  navigation  files  were  written  to  a  “final” 
data  directory  and  made  available  to  the  entire  shipboard  science  party. 

6.  Data  Archiving.  Regular  exabyte  tape  backups  were  made  of  the  processed  data. 

This  processing  procedure  was  followed  on  a  daily  basis.  The  author  performed  all  of 
the  processing  for  the  entire  survey,  ensuring  consistency  in  editing.  Navigation  accuracy 
is  described  in  (Lemer,  et  al,  1999). 

There  were  several  exceptions  to  the  absolute  routine  of  navigation  processing.  They 
were: 

1.  DSL-120  processing.  In  normal  practice,  the  sonar  processing  system  is  used  to 
handle  navigation  data  from  the  DSL-120.  However,  “teething  problems”  with 
sonar  software  made  the  sonar  system  unable  to  log  relay-based  LBL  navigation, 
so  the  standard  navigation  pipeline  was  used.  However,  difficulties  were  found  in 
obtaining  fixes  on  the  transponder  actually  mounted  to  the  sonar  tow  fish;  instead, 
a  transponder  mounted  on  the  main  cable  and  ahead  of  the  fish  was  used. 
Custom  software  to  compute  layback  corrections  based  upon  heading  and  the 
length  of  the  tether  between  the  clump  weight  and  the  fish  was  written,  and  this 
software  was  used  in  addition  to  that  described  above. 

2.  During  the  course  of  the  Argo  II  survey,  some  dissatisfaction  was  expressed  by 
members  of  the  EU  team  concerning  the  “jaggedness”  of  the  navigation  plot.  The 
jaggedness  resulted  from  the  use  of  real,  non-interpolated  data.  It  is  the  authors 
belief  that  in  the  absence  of  a  mathematical  model  fully  describing  the  dynamics 
of  the  underwater  vehicle,  the  “raw”  data  (with  outliers  removed)  is  the  best 
navigation  reference — even  if  the  track  which  results  is  not  “smooth.”  However, 
for  the  sake  of  aesthetic  plots,  some  gaussian  smoothing  was  done,  producing  a 
smooth  plot.  These  smoothed  data  were  always  clearly  labelled  and  separated 
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form  the  final  navigation  data  delivered  to  the  Assessors,  and  no  analysis  results 
were  based  upon  them. 

3.1.2.  ATTITUDE  PROCESSING 

1.  Data  Abstraction.  Attitude  data  was  part  of  the  data  flow  from  the  vehicle  topside 
processor.  Appendix  A  describes  all  of  this  data.  The  attitude  data  was  separated 
from  the  rest  of  the  data,  which  as  archived,  but  not  further  processed. 

2.  Data  Cleaning.  The  attitude  file  was  checked  for  records  out  of  order,  incomplete 
records,  and  extraneous  characters.  Given  the  simplicity  of  the  path  between  the 
vehicle  computer  system  and  the  data  logging  computer,  such  problems  rarely  if 
ever  occurred,  but  the  processing  step  remains  as  a  holdover  from  the  day  when 
inter-computer  communication  was  not  as  reliable  as  it  has  become. 

3.  Wild-Point  Editing:  A  simple  filter  examined  each  attitude  record  to  ensure  that  the 
data  vales  were  reasonable.  A  median  rejection  filter  was  used  to  examine  the 
data,  checking  each  point  to  be  sure  that  it  didn’t  differ  from  the  median  of  its 
neighbors  by  more  than  a  specified  amount.  Points  which  did  differ  significantly 
were  deleted  from  the  data  set. 

4.  Graphical  Editing.  The  attitude  data  set  was  plotted  in  a  custom  graphical  editor 
in  various  time  series  plots  (roll,  pitch,  heading,  depth,  and  altitude  versus  time. 
Data  points  thought  to  be  spurious  were  removed  during  this  process. 

5.  Data  Reorganization.  The  fully  processed  attitude  files  were  written  to  a  “final” 
data  directory  and  made  available  to  the  entire  shipboard  science  party. 

6.  Data  Archiving.  Regular  exabyte  tape  backups  were  made  of  the  processed  data. 

3.2.  SONAR  PROCESSING 

As  described  in  the  section  on  sonar  data  logging,  data  at  an  intermediate  stage  of 
processing  was  logged  across  the  network  to  a  hard  disk  on  another  workstation  (See 
Figure  4).  At  this  workstation,  several  kinds  of  processing  occurred: 

1 .  Navigation  data  derived  from  the  reprocessed  layback  data,  as  described 
previously,  was  re-merged  into  the  data  set. 

2.  Attitude  data  was  processed  using  tools  provided  in  the  sonar  processing  software. 
The  general  scheme  was  similar  to  that  described  in  Section  3.1.2,  but  there  were 
no  separate  files  produced.  Corrections  to  attitude  were  maintained  as  part  of  the 
sonar  data  record. 

3.  Sonar  data,  both  amplitude  and  bathymetry,  was  geo-located  using  the  navigation 
and  attitude  data,  and  placed  into  a  map  grid.  Multiple  lines  of  data  were  merged 
into  the  same  grid.  Small  mis-ties,  well  within  the  bounds  of  navigation  accuracy 
were  detected,  and  separate  maps  of  each  line  were  also  produced. 

4.  The  sonar  maps  were  made  available  to  the  science  party  and  used  for  planning 
Argo  and  Jason  operations. 
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3.3.  ELECTRONIC  STILL  CAMERA  PROCESSING 


Processing,  distributing,  and  archiving  the  ESC  data  collected  by  the  Argo  II  vehicle 
was  the  most  time  and  computer-resource  consuming  task  of  the  survey.  The  goal  of  the 
processing  was  to  get  high  quality  images  into  the  hands  of  the  assessors  and  the 
Inspection  and  Verification  (I  &  V)  team  as  soon  as  possible  after  the  tape  was  removed 
from  the  deck  box.  Therefor,  no  standard  daily  schedule  was  employed.  Processing 
continued  virtually  around  the  clock  during  the  entire  Argo  survey.  The  following  steps 
were  followed: 

1.  Setup  of  directories  and  files.  A  logical  system  of  file  organization  was  set  up.  As 
each  tape  was  processed,  a  new  set  of  directories  was  created,  minimizing  the 
chances  of  tapes  getting  confused  with  each  other. 

2.  Copy  from  tape  to  disk.  The  images  are  stored  on  tape  in  files  of  fifty.  Although 
this  is  wise  practice  when  using  tape  media  (a  tape  error  will  not  cause  the  loss  of 
more  than  one  file  at  a  time,  limiting  the  number  of  images  lost)  it  makes 
processing  cumbersome.  Therefor,  the  tape  was  read  all  at  once,  and  a  single  disk 
file,  containing  all  of  the  data  from  the  entire  tape  was  built.  The  raw  tape  was 
removed  from  the  processing  computer  and  returned  to  the  onboard  archive  at 
this  time. 

3.  Extract  images.  Due  to  a  peculiarity  in  the  way  MIS  chose  to  write  the  images  to 
tape,  a  block  of  binary  zeroes  had  to  be  removed  from  the  header  of  each  image 
before  the  image  itself  could  be  extracted.  This  was  done  for  each  image.  The 
date-time  string  was  extracted  from  each  image,  and  a  new  file  name  was  created. 
The  file  name  included  all  of  the  information  necessary  to  keep  each  image 
separate  from  any  other,  and  matched  the  following  pattern: 

ESC.YYMMDD_HHMMSS.IMNUM.mis 

"Where  YY  is  the  last  two  digits  of  the  year,  MM  is  the  month,  DD  is  the  day  of  the 
month,  HH  is  the  hour,  following  a  24  hour  system,  MM  is  the  minute,  and  SS  is 
the  second.  IMNUM  is  the  sequential  order  of  the  image  on  the  tape,  beginning 
with  zero,  mis  indicates  that  this  is  a  Marine  Imaging  Systems  format  file.  This 
same  file  name  pattern  was  followed  throughout  the  processing,  with  different 
format  type  suffixes  indicating  different  stages  of  processing  and  image  formats. 

A  new  image  time  file  was  also  created,  matching  the  image  name  and  its  time 

4.  Normalization.  At  this  point,  the  raw  image  was  still  in  a  custom  sixteen  bit  MIS 
format,  which  is  not  used  by  any  other  software  package.  Very  few  application 
software  packages  can  use  sixteen  bit  data  at  all.  The  goal  of  this  step  was  to 
optimally  convert  the  sixteen  bit  data  to  an  eight-bit  file  in  a  format  readily  useable 
by  other  software  packages. 

The  straightforward  way  to  convert  sixteen  bits  to  eight  is  to  re-map  the  histogram 
of  the  sixteen  bit  data  so  that  it  falls  into  256  (eight-bit)  values.  However,  two 
potentially  important  steps  are  missed  if  this  practice  is  followed. 

•  All  CCD  chips  have  flaws  that  show  up  as  artifacts  on  the  image.  In  particular, 
the  chip  used  in  the  MIS  camera  was  fabricated  in  a  two-stage  process,  and  the 
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chip  appears  to  have  two  distinct  left/right  halves.  Figure  4  shows  what  is 
called  a  bias  image.  It  represents  what  the  chip  reports  as  data  with  no 
exposure  to  light,  and  is  frequently  called  a  dark-current  image  (Newberry, 
1995).  The  two-half  effect  can  be  clearly  seen.  Also  seen  is  a  distinct  vertical 
striping,  which  is  related  to  the  manner  in  which  the  charge  is  read  out  of  the 
CCD.  The  majority  of  these  effects  are  removed  by  subtracting  a  bias  image 
from  each  raw  image  before  further  processing.  (Note  that  the  gray  scale 
ranges  of  the  bias  image  shown  if  Figure  4  have  been  stretched  so  as  to  make 
them  visible  for  this  report.  The  actual  image,  if  shown  unchanged,  would 
appear  almost  uniformly  black  due  to  the  very  low  pixel  values  recorded  in 
bias  images.) 


Figure  4:  ESC  Bias  Image 

•  Flat  fielding  is  an  essential  step  in  calibrating  a  raw  CCD  image.  It  is  necessary 
because  a  given  intensity  of  light  does  not  product  an  identical  response  in 
every  pixel  of  a  CCD  array  (Chromey,  1996).  Variations  occur  due  to 
sensitivity  differences  among  pixels  and  the  unique  characteristics  of  the 
optical  path  (among  other  causes).  The  unwanted  variations  are  removed  by 
dividing  a  raw  image  by  the  flat  field  frame.  The  flat  field  is  obtained  by 
exposing  the  CCD  chip  to  a  range  of  grey  light  fields,  and  picking  the  one 
which  most  closely  matches  the  expsosures  obtained  by  the  images  bing 
adjusted. 

These  two  processes  are  combined  in  a  step  called  normalization.  The  output  is 
an  eight-bit  image  that  accurately  represents  the  scene  imaged  by  the  camera, 
which  has  very  little  artifacts  induced  by  the  camera.  All  of  the  Derbyshire  survey 
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images  were  normalized.  The  normalized  images  were  distinguished  by  a  suffix  of 
“.if"  indicating  a  sun  raster  file. 

5.  Histogram  Specification.  There  were  two  primary  ESC  image  products  during  the 
survey;  individual  images  for  interpretation,  and  photo-mosaics.  Histogram 
specification  was  key  to  both  of  these. 

Underwater  imagery  is  usually  characterized  by  uneven  illumination.  Shadows  can 
be  quite  useful  in  interpretation,  since  they  reveal  differences  in  height  and  aspect. 
Illumination  falloff,  however,  shows  only  the  physics  of  light  propagation  in  the 
water:  the  farther  light  travels,  the  more  it  is  attenuated  and  lost.  Furthermore,  the 
uneven  illumination  causes  a  mottled  mosaic,  and  detracts  greatly  from  the 
finished  product.  The  eyes  tend  to  be  drawn  to  the  imperfections  and  apparent 
edges  caused  by  illumination  differences,  rather  than  freely  viewing  the  entire 
mosaic.  A  processing  step  that  manipulates  the  gray  scale  values  in  an  image  to 
produce  apparently  even  illumination  is  invaluable  in  adding  both  interpretation 
and  mosaicking. 

WHOI  has  been  using  adaptive  histogram  equalization,  and  its  cousin,  adaptive 
histogram  specification  for  many  years  to  successfully  meet  this  need.  Basically, 
each  image  is  divided  into  contiguous  small  blocks  of  pixels.  The  histogram  of 
each  block  is  calculated,  and  either  equalized  or  passed  through  the  transfer 
function  of  the  desired  distribution.  Then  for  each  pixel  of  the  input  image,  a 
weighted  average  of  the  transferred  histograms  of  the  surrounding  blocks  is 
calculated,  and  a  new  pixel  value  computed  (Pratt,  1991). 

Underwater  surveys  typically  collect  many  thousands  of  images.  It  is  entirely 
impractical  to  individually  process  each  image,  so  a  batch  method  of  processing 
was  used  during  the  Derbyshire  survey.  Parameters  for  the  histogram  specification 
were  based  upon  DSL  experience  and  upon  imagery  collected  during  the  first 
several  thousand  ESC  images;  these  parameters  were  then  used  to  process  the 
entire  data  set.  Custom  software  written  by  the  author  was  used  for  this 
processing  step.  The  output  of  the  processing  was  an  eight-bit  Sun  raster  file. 

There  are  several  parameters  which  can  be  varied  in  the  histogram  specification 
process.  The  Woods  Hole  software  allows  modification  of  the  following: 

•  Type  of  histogram.  The  choices  in  the  WHOI  software  are  none  (which  leaves 
the  image  unmodified),  uniform  (which  is  identical  to  adaptive  histogram 
equalization),  rayleigh  (which  was  used  for  the  Derbyshire  survey)  and 
exponential,  which  was  implemented  as  an  experiment,  and  has  never,  in 
WHOI  experience,  produced  results  as  good  as  the  rayleigh  distribution. 
There  is  no  theoretical  justification  for  use  of  the  rayleigh  distribution,  but  it 
has  been  shown  experimentally  to  be  satisfactory. 

•  Alpha.  The  rayleigh  and  exponential  distributions  require  specification  of  an 
“alpha”  parameter,  which  modifies  the  specified  distribution.  Values  are 
chosen  experimentally,  and  must  fall  between  0.0  and  1.0. 

•  Region  Count.  This  parameter  specifies  the  number  of  contextual  regions 
which  will  be  used  in  histogram  manipulation.  If  this  number  is  chosen  too 
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high,  noise  and  small  texture  changes  will  be  exaggerated.  If  it  is  chosen  too 
low,  illumination  gradients  will  not  be  eliminated  as  effectively. 

In  order  to  show  that  the  parameters  chosen  were  reasonable,  six  of  the  images 
collected  during  the  survey  have  been  chosen  for  analysis.  Appendix  C  presents 
these  images,  processed  with  a  variety  of  different  parameters.  In  retrospect,  the 
parameters  chosen  during  the  Derbyshire  survey  produced  the  most  uniformly 
high  quality  results  yet  seen  during  DSG  surveys. 

6.  Format  conversion.  In  order  to  make  the  data  more  widely  useable  (Sun  raster 
files  are  not  widely  used  on  PC  and  Macintosh  computers)  the  histogram  specified 
files  were  converted  to  TIFF  files.  This  conversion  happened  with  no  loss  of 
resolution  or  dynamic  range,  it  was  really  just  a  rearranging  of  the  data.  The  file 
names  of  the  tiff  files  followed  the  same  format  as  described  previously,  with  the 
exception  that  the  file  names  ended  in  “tif.” 

7.  Data  Distribution.  After  processing,  all  of  the  histogram-specified  images  were 
copied  to  Intel/Windows  NT  workstations  for  analysis  and  use  in  mosaicking. 
They  were  kept  on-line  as  long  as  possible,  and  in  no  case,  were  they  ever 
removed  without  approval  of  the  I  &  V  Team.  Before  removal,  they  were  copied 
to  JAZ  disks  for  ready  access  in  later  analyses  and  mosaicking  efforts. 

8.  Archival.  Data  from  five  raw  ESC  tapes  and  their  associated  data  products  were 
archived  to  exabyte  tapes.  All  of  the  important  results  of  processing  were  saved; 
the  raw  tapes  from  the  deck  box,  the  raw  MIS  format  images,  the  normalized 
images,  and  the  histogram  specified  images.  Two  copies  of  each  archival  tape 
were  made;  one  for  transferal  to  the  U.K.  and  one  for  use  at  Woods  Hole. 

9.  Coverage  charts.  The  files  containing  ESC  time  stamps  were  merged  with  Argo 
position  and  attitude  data  to  produce  a  “merge”  record  that  contained  all  the 
information  necessary  to  describe  the  position  and  orientation  of  the  camera  at  the 
instant  of  exposure,  along  with  its  altitude.  These  records  were  used  in  custom 
DSG  software  to  produce  coverage  charts  that  were  used  to  monitor  progress  and 
plan  operations.  The  ground  coverage  of  each  image  was  computed  using  vehicle 
altitude  and  other  camera  parameters,  and  plotted  on  a  large-scale  basemap  of  the 
survey  area.  The  final  version  of  this  report  appears  in  (Williams,  et  al,  1998). 

The  processing  stream  just  described  was  followed  for  most  of  the  ESC  tapes.  There 
were  several  exceptions,  caused  by  occasional  tape  errors.  In  these  cases,  the  batch 
processing  would  be  interrupted  by  the  errors,  and  “hand”  processing  would  be  used. 
In  all  “hand”  processing,  the  same  image  normalization  and  histogram  specification 
parameters  were  used;  the  only  difference  is  that  each  processing  step  would  be 
initiated  by  an  operator  instead  of  by  the  computer.  This  “hand”  processing  of  tape  or 
image  errors  was  able  to  recover  virtually  all  of  the  data  which  produced  errors  during 
the  batch  processing.  Image  data  produced  in  this  way  is  indistinguishable  from  that 
produced  during  the  more  routine  batch  processing. 
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4.  DELIVERY  OF  DATA  PRODUCTS 


Throughout  the  survey,  the  results  of  the  serial  data  processing  just  described  were 
available  to  the  entire  survey  party  in  a  directory  structure  on  a  DSG  computer.  The  data 
was  used  frequently  by  both  DSG  and  UK/EU  personnel.  Copies  of  the  final  data  were 
copied  onto  exabyte  tape  and  provided  to  the  Assessors  in  a  Unix  tar  format. 
Additionally,  copies  of  the  data  were  placed  onto  the  workstation  which  was  delivered  to 
the  UK  DETR,  and  also  onto  JAZ  disks,  which  were  turned  over  to  the  Assessors.  Copies 
were  also  held  by  WHOI.  These  copies  were  used  post-cruise  to  produce  Compact  Disks 
for  further  archiving  and  delivery. 

Copies  of  all  of  the  raw,  normalized,  and  histogram-specified  ESC  imagery  were  made 
to  exabyte  tape  and  turned  over  to  the  Assessors.  Additionally,  a  complete  set  of 
histogram-specified  data  was  delivered  on  either  the  workstation  hard  disk  or  on  JAZ  disk. 
Copies  were  also  held  by  WHOI;  these  were  used  for  making  copies  to  compact-disk  for 
later  transferal  to  the  DETR. 

An  analysis  system,  capable  of  viewing  and  accessing  all  of  this  data  was  delivered  to 
DETR  after  the  survey.  The  principal  component  of  this  analysis  system,  the  Visual 
software  package,  is  documented  in  (Lemer,  1999). 
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APPENDIX  A:  DATA  FORMATS 


This  Appendix  describes  the  formats  of  the  data  collected  and  processed  by  the 
Unmanned  Vehicles  of  the  WHOI  Deep  Submergence  Group  during  the  1997  Derbyshire 
Survey. 


A.l.  SERIAL  DATA  FORMATS 

Standard  DSG  data  strings  are  logged  as  ASCII  text  strings  that  are  terminated  with 
“\n”  (ODOA  hex).  The  record  has  the  following  general  format: 

<STRING  TYPE>  <DATE/TIME  RECORD>  <DATA  SOURCE>  <DATA  FIELDS>  <\n> 

where  the  individual  fields  are  as  follows: 

•  <STRING  TYPE> 

The  first  field  of  every  standard  DSG  data  string  is  the  string-type  field,  a  3-letter  code 
indicating  the  type  of  the  data  string.  The  types  presently  in  use  include: 

♦  PAS:  Platform  Attitude  Status  -  containing  vehicle  heading,  roll,  pitch,  depth, 
and  altitude  data. 

♦  PNS:  Platform  Navigation  Status  -  containing  vehicle  position  data. 

♦  PGE:  Platform  Gyro  Event  -  containing  gyro  events  such  as  pilot  initiated 
resets  of  gyro  to  current  magnetic  heading,  and  gyro  power-on  events. 

♦  JTS:  Jason  Thruster  Status  containing  current  Jason  thruster  command  values. 

♦  JSS:  Jason  Switch  Status  -  containing  current  values  of  various  Jason  on-board 
switches  such  as  lights  and  other  power  controls. 

♦  VID:  A  Video  record,  containing  events  pertaining  to  video  tape  start,  stop, 
and  pause  times.  These  records  were  an  experiment  during  the  Derbyshire 
survey  period,  and  are  no  longer  used  in  the  Deep  Submergence  Group 

•  <DATE/TIME  RECORD> 

The  second  field  of  every  standard  DSG  data  string  is  a  date/time  field  that  documents 
the  Greenwich  time  at  which  the  data  was  logged.  It  is  in  the  format  YY/MM/DD  HH: 
MM:  SS.SS. 

♦  YY:  Year,  with  no  century  indication 

♦  MM:  Month 

♦  DD:  Day  of  month,  with  leading  zero  as  appropriate 

♦  HH:  Hour,  on  a  24  hour  clock,  leading  zero  as  appropriate 

♦  MM:  Minutes  past  the  hour,  leading  zero  as  appropriate 
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♦  SS  Seconds,  in  decimal  notation  to  0.01  second  resolution,  leading  zero  as 
appropriate 

•  <DATA  SOURCE> 

The  third  field  of  every  standard  DSG  data  string  is  the  data  source  field  -  indicating 
that  the  data  is  either  from  or  about  a  particular  vehicle  or  sensor. 

•  JAS  Data  was  generated  by  Jason  or  Argo.  Note  that  ARGO  and  JASON  generate 
identical  log  strings.  Both  are  referred  to  as  "JAS"  in  the  raw  log  files. 

•  MED  Data  was  generated  for  Medea 

•  IMA  Imagenix  scanning  sonar 

•  SHP  The  surface  ship  position 

•  REF  The  surface  ship  reference  position 

•  GPS  The  surface  ship's  position  according  to  GPS 

•  WRN  The  surface  ship's  position  according  to  the  WRN  Military  GPS  reciever 

•  LBL  The  LBL  navigation  system 

•  <DATA  FIELDS> 

The  remaining  fields  of  every  standard  DSL  data  string  contain  numerical  data 
delimited  by  white  space. 

•  <\cr> 

The  last  character  of  every  standard  DSL  data  string  is  the  ASCII  string  termination 
character  “\n”  (0D0A  hex). 


A.2.  SPECIFIC  DATA  RECORD  FORMATS 

A.2.1.  PNS  Records 

PNS  records  are  comprised  of  at  least  9  fields: 

1.  "PNS" 

2.  Date  string:  YY/MM/DD 

3.  time  string:  HH:MM:SS.SS 
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4.  source 

5.  coordinate  frame 

6.  vehicle 

7.  X  position 

8.  Y  position 

9.  Z  position 

10.  Quality  Indicator  (in  some  records) 

11.  Transponder  Pair  Indicator  (in  some  records) 

The  Source  field  identifies  the  origin  of  the  record.  It  can  take  on  many  forms,  including 
WRN,  which  indicates  a  GPS  receiver  of  some  type,  GPS,  which  also  indicates  a  satellite 
receiver  and  LBL,  which  indicates  an  acoustic  navigation  fix.  Navigation  records  use  one 
of  three  coordinate  frames  in  the  fifth  field:  GLL,  NEN,  or  UTM.  GLL  is  the  Geodetic 
Latitude  and  Longitude  indicator.  Latitude  or  Longitude  is  represented  as  decimal  degrees, 
Positive  North  and  East.  NEN  stands  for  Net  East-North,  a  local  navigation  frame  used  in 
Long  BaseLine  (LBL)  navigation.  However,  during  the  Derbyshire  survey,  UTM 
coordinates  were  logged  in  NEN  records.  UTM  stands  for  Universal  Transverse  Mercator, 
a  worldwide  (except  for  very  high  latitudes)  projection  system  that  the  DSG  uses  for 
almost  all  processing  and  mapping.  The  Derbyshire  survey  was  carried  out  almost 
entirely  in  UTM  Zone  53.  Units  are  in  meters.  (A  description  of  UTM,  which  includes 
equations  for  the  Latitude-Longitude<->UTM  transformation,  is  found  in  (Evenden,  1990  ). 

The  vehicle  field  identifies  what  is  being  navigated.  Possible  identifiers  include 

•  SHP:  The  surface  vessel 

•  JAS:  The  Jason  vehicle 

•  ARG:  The  Argo  vehicle 

•  MED:  The  Medea  vehicle 

•  FSH:  generally,  the  DSL-120  sonar 

•  EMG:  The  emergency  transponder  botde 

•  RLY:  A  relay  transponder 

The  last  two  identifiers  can  be  used  to  navigate  virtually  any  of  the  DSG  vehicles,  and 
have  to  be  considered  in  context,  knowing  which  vehicle  was  in  the  water  at  what  time. 
The  navigator  on  watch  shifts  frequencies  as  necessary  to  safely  navigate  the  vehicle  in 
real  time. 
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The  Z  field  of  the  navigation  coordinates  represents  either  the  depth  of  the  ship 
transducer  (in  a  SHP  LBL  record)  or  the  calculated  acoustic  depth  of  the  vehicle  being 
navigated. 

Sample  PNS  Strings: 

PNS  97/04/01  23:59:55.54  WRN  GLL  SHP  25.862668  133-532262  0.00  1.2  00 

PNS  97/04/01  23:59:55.54  GPS  UTM  SHP  352931.77  2861297.19  0.00  1.2  06 

PNS  97/04/01  23:59:52.00  LBL  NEN  SHP  352963.41  2861276.11  5.80  0.0  00 

PNS  97/04/01  23:59:57.58  WRN  GLL  SHP  25.862663  133-532267  0.00  1.2  00 

PNS  97/04/01  23:59:57.58  GPS  UTM  SHP  352932.26  2861296.63  0.00  1.2  06 

A.2.2.  PAS  records 

There  are  two  kinds  of  PAS  records  in  the  data  set.  The  first  is  a  vehicle-based  record, 
which  is  comprised  of  15  fields: 

1.  "PAS" 

2.  Date  string:  YY/MM/DD 

3.  Time  String:  HH:MM:SS.SS 

4.  "JAS" 

5.  heading,  Humphrie’s  mechanical  gyro,  degrees,  true  (corrected  for  local 
variation) 

6.  heading,  KVH  magnetic  compass,  degrees,  true  (corrected  for  local  variation) 

7.  pitch,  inclinometer,  degrees 

8.  roll,  inclinometer,  degrees 

9-  altitude,  Datasonics  acoustic  altimeter,  meters  above  bottom 

10.  depth,  Paroscientific  pressure  transducer,  meters  below  surface 

11.  gyro  correction  for  Humphrie’s  mechanical  gyro,  degrees 

12.  local  magnetic  variation,  degrees 

13.  pitch,  Watson,  degrees 

14.  roll,  Watson,  degrees 

15.  0.0 

16.  0.0 

The  second  type  of  PAS  record  is  found  in  the  raw  navigation  files,  and  is  a  ship’s  gyro 
record.  It  is  comprised  of  ten  fields: 

1.  "PAS" 

2.  Date  string:  YY/MM/DD 

3.  Time  String:  HH:MM:SS.SS 

4.  "SHP" 

5.  heading,  ship’s  gyro 


23 


6.  0.0 

7.  0.0 

8.  0.0 

9.  0.0 

10.  0.0 

The  last  five  fields  were  always  set  to  zero  for  consistency  with  the  vehicle-originated  PAS 
format  and  potential  use  in  other  DSG  processing  software. 

All  magnetic  headings  are  logged  in  degrees  of  true  heading.  Magnetic  headings  are 
corrected  to  true  headings  by  applying  the  local  magnetic  variation  programmed  that  is 
programmed  into  the  Jason/Argo  topside  computer.  The  sign  convention  of  magnetic 
variation  is 


Negative  =  west,  positive  =  east. 

Raw  magnetic  heading  is  converted  to  true  heading  with  the  formula: 

True  heading  =  raw  heading  +  current  magnetic  variation; 

Gyro  heading  is  logged  in  degrees  true.  The  gyro  is  corrected  for  earth-rate  rotation 
based  on  the  latitude  programmed  into  the  Jason/Argo  topside  computer.  The  gyro  drifts, 
sometimes  over  10  degrees  per  hour.  The  pilot  periodically  manually  resets  the  gyro 
heading  to  the  current  true  heading  indicated  by  the  magnetic  compass. 


Sample  PAS  Strings: 

PAS  97/06/24  16:39:00.07  JAS  350.6  352.3 

0.0 

PAS  97/06/2 4  16:39:00.83  JAS  350.6  352.0 

0.0 

PAS  97/06/24  16:39:01.11  JAS  350.7  352.0 

0.0 

PAS  97/06/24  16:39:01.42  JAS  350.8  351.5 

0.0 

PAS  97/06/24  16:39:01.89  JAS  351.0  351.5 

0.0 


-1.9 

5.2 

4.2 

845.028 

-23.6 

1.0  -3-9066  3.4323 

0.0 

-2.5 

5.0 

4.2 

845.015 

-23.6 

1.0  -4.3954  3.2968 

0.0 

-2.8 

5.1 

4.2 

845.015 

-23.6 

1.0  -4.6408  3.2236 

0.0 

-2.9 

5.0 

4.2 

845.015 

-23.6 

1.0  -4.6591  3.1796 

0.0 

-3.2 

4.9 

4.2 

845.035 

-23.6 

1.0  -4.5273  3.1796 

0.0 

A.2.3.  JTTS  Records 


JTS  records  are  not  used  in  routine  processing,  but  are  logged  and  maintained  for  use 
inthruster  analysis.  They  are  comprised  of  seventeen  fields: 

1.  "JTS" 

2.  Date  string:  YY/MM/DD 

3.  Time  string:  HH:MM:SS.SS 

4.  "JAS" 
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5.  Port  Horizontal  (PH)  thruster  command  in  Newtons 

6.  Starboard  Horizontal  (SH)  thruster  command  in  Newtons 

7.  Forward  Lateral  (FL)  thruster  command  in  Newtons 

8.  Aft  Lateral  (AL)  thruster  command  in  Newtons 

9-  Port  Vertical  (PV)  thruster  command  in  Newtons 

10.  Starboard  Vertical  (SV)  thruster  command  in  Newtons 

11.  Aft  Vertical  (AV)  thruster  command  in  Newtons 

12.  7  bit  bitfield  indicating  power  status  of  each  vehicle  thruster  as  follows: 

PH  power  on?  (0  or  1) 

SH  power  on?  (0  or  1) 

FL  power  on?  (0  or  1) 

AL  power  on?  (0  or  1) 

PV  power  on?  (0  or  1) 

SV  power  on?  (0  or  1) 

AV  power  on?  (0  or  1) 

13.  U  Thrust  command.  Total  commanded  thrust  in  forward  direction,  Newtons. 

14.  V  Thrust  command.  Total  commanded  thrust  in  starboard  direction,  Newtons. 

15.  W  Thrust  command.  Total  commanded  thrust  in  down  direction,  Newtons. 

1 6.  Heading  Moment  command.  Total  commanded  moment  in  clockwise  direction, 
Newton-meters. 

17.  Jason-Argo  configuration: 


0  =  Jason  deep  configuration  -  1HP  Thrusters  as  forward  verticals 

1  =  Jason  shallow  configuration  -  1HP  Thrusters  as  laterals 

2  =  Argo  configuration  -  1HP  Thrusters  as  laterals 


Sample  JTS  Strings: 


JTS  98/04/20 

10:00:01.58  JAS 

24.6804 

13.5970 

5.5417 

-2.7708  -14.3563 

-14.3563  - 

7.1781  1111111 

35.4901  2.7708 

-35.8907 

10.5846 

0 

JTS  98/04/20 

10:00:02.58  JAS 

34.9563 

3.3211 

15.8176 

-7.9088  -14.3563 

-14.3563  - 

7.1781  1111111 

35.4901  7.9088 

-35.8907 

30.2116 

0 

JTS  98/04/20 

10:00:03.59  JAS 

29.0642 

9.2131 

9.9255 

-4.9628  -14.3563 

-14.3563  - 

7.1781  1111111 

35.4901  4.9628 

-35.8907 

18.9578 

0 
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A.2.4.  JSS  Records 


JSS  records  are  not  processed  during  normal  operations.  They  represent  system  status, 
and  are  comprised  of  ten  fields: 

1.  "JSS" 

2.  Date  string:  YY/MM/DD 

3.  Time  string:  HH:MM:SS.SS 

4.  'JAS" 

5.  ten-bit  bitfield  indicating  the  power  status  of  the  following  devices:  (0=off, 
l=on) 

•  pan-tilt 

•  gyro  compass 

•  magnetic  compass 

•  pressure  depth  sensor  (Paroscientific) 

•  acoustic  altimeter 

•  manipulator 

•  electronic  still  camera 

•  Imagenex  sonar 

•  sidescan  sonar 

6.  four-bit  bitfield  indicating  the  power  status  of  the  following  devices:  (0=off, 
l=on) 

•  light  circuit  1 

•  light  circuit  2 

•  light  circuit  3 

•  light  circuit  4 

7.  four-bit  bitfield  indicating  the  power  status  of  the  following  devices:  (0=off, 
l=on) 

•  video  circuit  1 

•  video  circuit  2 

•  video  circuit  3 

•  video  circuit  4 
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8.  two-bit  bitfield  indicating  the  power  status  of  the  following  devices:  (0=off, 
l=on) 

•  science  bus  1 

•  science  bus  2 


9-  A  two-integer  field  indicating  the  state  of  the  two  subsea  video  switches.  Two 
of  the  video  channels  from  subsea  to  topside  are  selectable  via  two  subsea 
video  switches  to  any  of  four  possible  subsea  cameras.  The  two  digits  of  this 
field  indicate  the  switch  state  of  these  two  video  switches. 


10.  Integer  frame  count  for  the  35mm  film  camera. 


Sample  JSS  Strings: 

JSS  98/04/22  08:06:35.68  JAS  0101101100  0000  1111  00  31  0 
JSS  98/04/22  08:06:45.68  JAS  1101101100  0000  1111  11  31  0 
JSS  98/04/22  08:06:55.69  JAS  1101101100  0010  1111  11  31  0 
JSS  98/04/22  08:07:05.71  JAS  1101101100  0010  1111  11  31  0 

A.2.5.  VED  Records 

VID  records  are  comprised  of  5  fields: 

1.  "VID'' 

2.  Date  string:  YY/MM/DD 

3.  Time  string:  HH:MM:SS.SS 

4.  "VID" 

5.  Event  string,  in  the  form  ACTION-DECK_TYPE_DECK_NUMBER,  where  action 
is  STOP  or  REC,  deck_type  is  HI8  or  HIRES,  and  the  number  is  a  sequential 
number  assigned  at  installation. 

VID  records  were  created  by  a  Crestron  control  panel,  which  is  a  digital  device  often  used 
to  control  devices  such  as  tape  decks  in  auditoriums  and  other  facilities.  WHOI  used  a 
Crestron  as  an  experimental  way  to  control  the  multitude  of  video  decks  which  were  used 
during  the  Derbyshire  suyrvey,  but  has  not  continued  the  practice. 
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A.3  ELECTRONIC  STILL  CAMERA  DATA  FORMATS 


The  electronic  still  camera  images  which  were  delivered  to  DETR  were  in  several 
formats.  The  first  is  the  custom  Marine  Imaging  Systems  format.  Definition  of  this  format 
is  available  from  the  author. 

The  next  two  formats  are  sun  raster  flies  and  tiff  files.  Ample  descriptions  of  these 
formats  can  be  found  in  (Murray  and  VanRyper,  1994)  or  any  other  graphics  reference. 


28 


A.4  SONAR  DATA  FORMATS 


Definitions  of  the  sonar  data  format  is  available  from  the  author  or  from  the  vendor  of 
the  software  used  for  real-time  collection  and  post  processing  (Oceanic  Imaging 
Consultants,  Honolulu,  Hawaii). 
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APPENDIX  B:  GLOSSARY  OF  ACRONYMS  &  TERMS 


Amplitude  Data:  The  term  used  to  describe  acoustic  backscatter  strength  received  at 
a  side  scan  sonar  receiver.  Compare  to  Bathymetry  data. 

Attitude  Data:  vehicle  roll,  pitch,  and  heading  data 

Bathymetry  Data:  Measurements  of  the  depth  of  the  sea  floor.  Used  in  this  report  to 
describe  one  type  of  data  from  the  DSL-120  side-scan  sonar. 

CCD:  Charge  Coupled  Device,  the  imaging  system  used  in  an  electronic  still  camera 

Chronolog:  A  precise  clock  used  as  a  time  standard  in  the  DSG  control  van. 

Clump:  a  heavy  weight  attached  to  the  main  fiber  optic  cable,  behind  which  the  DSL- 
120  is  towed  using  a  flexible  cable. 

CRV:  Continuous  Recording  Velocity,  a  type  of  laser  video  disk. 

Data  Logging  Workstation:  One  of  two  Sun  Sparc  LX  workstations  used  in  the 
control  van  to  collect,  store,  and  distribute  serial  data. 

Data  Processing  Workstation:  A  Sun  Sparc  LX  workstations  which  was  located  in 
the  computer  Laboratory  and  was  used  to  process  Serial  data,  as  well  as  ESC  data.  The 
Sonar  Processing  workstation  was  also  used  for  these  tasks  when  not  busy  processing 
sonar  data 

DETR:  The  Department  of  the  Environment,  Transport  and  the  Regions. 

DSG:  Deep  Submergence  Group. 

DSL:  The  Deep  Submergence  Laboratory 

ESC:  Electronic  Still  Camera,  a  still  camera  which  uses  digital  storage  of  its 
information.  Can  be  compared  to  a  film  camera,  which  uses  chemical  media. 

Exabyte:  a  type  of  data  storage  tape. 

JAZ:  A  type  of  magnetic  storage  media  made  by  the  Iomega  Corporation. 

Layback:  A  term  used  to  describe  the  distance  at  which  a  towed  vehicle  follows 
either  a  surface  ship  or  a  depressor  weight. 

LBL:  Long  Baseline  Acoustic  navigation  system. 

Macintosh:  A  type  of  personal  computer 

MIS:  Marine  Imaging  Systems,  the  vendor  who  produced  the  ESC  used  during  the 
Derbyshire  survey.  A  subsidiary  of  the  Marquest  Group. 
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Navigation/Control  Computer:  An  IBM  compatible  PC  used  to  display  and  transmit 
data  from  all  the  navigation  systems  used  on  the  ship/vehicle  systems,  and  to  send  data  to 
both  the  ship  and  vehicle  control  systems 

Portmaster:  A  device  made  by  Livingston,  Inc,  which  multiplexes  RS-232  data  onto 
an  ethemet  stream. 

Serial  Data:  for  the  purposes  of  this  report,  that  data  which  is  represented  as  a  time 
series,  and  sent  to  a  data  logging  computer  over  an  RS-232  line. 

SGI:  Silicon  Graphics  Incorporated,  a  vendor  of  Unix-based  workstations  frequently 
used  for  data  visualization. 

SMPTE:  The  Society  of  Motion  Picture  and  Television  Engineers,  the  promulgator  of 
standards  for  video  systems. 

Sonar  Workstation:  One  of  two  Sparc  workstations  used  to  collect  and  process  data 
from  the  DSL-120  and  200  sonar  systems.  One  of  these  workstations  was  located  in  the 
control  van  (the  Sonar  Logging  workstation)  and  the  other  in  the  Main  Lab  (The  Sonar 
Processing  workstation). 

Sparc:  a  specific  computer  architecture  marketed  and  sold  by  Sun  Microsystems.  The 
data  logging,  processing,  and  sonar  collection  workstations  were  all  spare-based. 

Sun:  Sun  Microsystems,  a  vendor  of  Unix-based  workstations 

Tar:  A  tape  archiving  format  used  primarily  in  the  Unix  operating  system. 

Unix:  A  type  of  computer  operating  system 

UTM:  Universal  Transverse  Mercator,  a  map  projection  system. 

Vehicle  Topside  Computer:  An  embedded,  transputer  based  processor  which 
communicates  with  subsea  computing  systems  on  both  Jason  and  Argo,  and  routes  data 
between  those  subsea  systems  and  other  topside  systems  which  need  the  data. 

WHOI:  The  Woods  Hole  Oceanographic  Institution 


APPENDIX  C:  ELECTRONIC  STILL  CAMERA  PROCESSING 


.This  appendix  shows  examples  of  processing  of  electronic  still  camera  imagery  in 
a  number  of  different  ways.  For  each  set,  the  normalized  image  is  shown  in  (a).  Second, 
the  image  as  processed  during  the  survey  is  shown  0?).  Then,  the  results  of  passing  the 
normalized  images  through  alternate  sets  of  parameters  are  shown  (c),  (d),  and  (e).  The 
parameters  used  in  the  processing  are  indicated  in  the  caption,  where  n  indicates  the 
number  of  contextual  regions,  t  indicates  the  histogram  type  (0  is  none,  1  is  uniform,  2  is 
exponential,  and  3  is  Rayleigh),  and  a  indicates  alpha.  A  simple  clip  with  a  limit  of  5  was 
used  for  all  the  images. 


Tape  007,  Image  455 


a)  Normalized  Image  455 
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d)  Image  455,  n=12,  t  =  3,  a=0.9 
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e)  Image  455,  n-3,  t  =  2,  a=0.9 
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Tape  025,  Image  408 
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c)  Image  408,  n=3,  t  =  3,  a=0.3 
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d)  Image  408,  n=12,  t  =  3,  a=0.9 


e)  Image  408,  n=3,  t  =  2,  a=0.9 
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Tape  031,  Image  779 


a)  Normalized  Image  779 
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c)  Image  779,  n=3,  t  =  3,  a=0.3 


d)  Image  779,  n=12,  t  =  3,  a=0.9 
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e)  Image  779 \  n=3,  t  =  2,  a=0.9 


f)  Image  779,  n=3,  t  =  1,  a=0.9 
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Tape  071,  Image  1530 


a)  Normalized  Image  1530 
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c)  Image  1530,  n=3,  t  =  3,  a=0.3 
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f)  Image  1530,  n=3,  t=l,  a=0.9 
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Tape  095,  Image  1377 


a)  Normalized  Image  1377 
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b)  Image  1377  as  processed  during  survey  (n=3,  t  =  3,  a-0.6) 
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e)  Image  1377,  n=3,  t  =  2,  a=0.9 
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Tape  098,  Image  362 


a)  Normalized  Image  362 


b)  Image  362  as  processed  during  survey  (n=3f  t  =  3,  a-0.6) 


c)  Image  362,  n=3,  t  =  3,  a-0.3 


d)  Image  362,  n=12,  t  =  3,  a=0.9 
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e)  Image  362,  n=3,  t  -  2,  a=0.9 
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f)  Image  362,  n=3,  t  =  1,  a=0.9 
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